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ABSTRACT 



This paper briefly describes the processing performed in the course 
of producing a line drawing from an image obtained through an isage 
dissector camera. The edge-marking phase uses a non-linear 
parallel line-follower. Complicated statistical measures are not 
used. The line and vertex generating phases use a number of 
heuristics to guide the transition from edge-fragments to cleaned- 
up line-drawing. Higher-level understanding of the blocks-world is 
not used. Sample line-drawings produced by the program are 
Included. 



Work reported herein was conducted at the Artificial Intelligence 
Laboratory, a Massachusetts Institute of Technology research 
program supported in part by the Advanced Research Projects Agency 
of the Department of Defense and monitored by the Office of Naval 
Research under Contract Number N00014-70-A-0362-0005. 

Reproduction of this document, in whole or in part, is permitted 
for any purpose of the United States Government. 

• This is a revised version of Vision Flash 16 (July 1971). 
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Programs able to produce line-drawings from luges of single 
convex polyhedra have existed for several years. It was thought 
that it should be easy to generalize the methods used to deal with 
more complex blocks-world scenes Involving obscuration, shadows and 
mutual Illumination. Only recently, however, has It become 
possible to reliably produce line-drawings of images of sets of 
polyhedra. There are two main problems. Firstly, images of sets 

of polyhedra are not as simple as one might expect. Secondly, 

■ 

current scene-analysis programs demand a complete line-drawing with 

well-defined vertices. 

* + 

One might reasonably conjecture that the image of a set of 
polyhedra ought to consist of polygonal areas of more or less 
uniform intensity. These areas should be separated by step-like 
transitions in intensity at the lines corresponding to the 
projections of the edges of the objects. This is not the case 
because of mutual illumination, shadowing, scattering of light, 
surface smudges, translucency and a number of defects in the 
Imaging device (Norn 19691. Usually in fact, the variation In 
intensity within one region (corresponding to a face or the visible 
portion of a face of an object) is often larger than the difference 
between the intensity of adjacent regions. Furthermore, only the 
most obvious edges (such as those separating the scene from the 



/ 



back-ground) are associated with anything like a step-like 
transition In intensity. Many transitions are instead roof-shaped 
or flat except for a snail peak right on the edge [Herskovits $ 

Binford 1970], 

■ * ■ 

+ 
The most obvious image degrading effect Introduced by the 

sensing device Is noise* In our case this amounted to about 1 to 

2% of the signal magnitude [Horn 1969). There is little point 

trying to improve on this, since the surface visual noise Is not 

much smaller even for clean, evenly painted polyhedra. One of many 

other device defects is internal scattering, which reduces the 

contrast between adjacent regions by alnost a factor of two and 

produces annoying gradients within regions [Horn 19721. 

2^ CLASSIFICATION OF LINE-FINDERS. 

Line-finders can be classified according to whether they are 
region-oriented or edge-oriented- Region-oriented line-finders 
attempt to find regions of more or less homogeneous intensity. The 
areas not conglomerated in this fashion are then thinned out and 
lines are fitted to them. Edge-oriented line-finders avoid the 
problems of inhomogeneous regions and the blending together of 
adjacent regions. They Instead determine likely edge-points and 
link these up into edge-fragments. The fragments are later 
combined into lines. 






Line-flnders can also be classified according to whether or 
not the filter function (also called the local predicate) applied 
to the Image Is linear* If visual noise were spatially 
Independent, a case could be made for an optimal linear predicate. 
This however Is not the case, since smudges, for example, have a 
distinctly non-random spatial distribution. Non-linear methods, 
while more complex, have a clear advantage on real Images. 

Llne-flnders can be further classified according to whether 
they use a fixed raster scan or a search pattern which follows 
lines. The first method Is convenient from the point of view of 
reading intensity values asynchronously ahead of time and also for 
program simplicity. In some cases the imaging device forces the 
use of a raster scan, particularly If It Is fln integrating device. 
Line-followers on the other hand can be made to be more sensitive 
and accurate at the price of program complexity. 

L_ EDGE-MARKING. 

■ , * 

Input is obtained from a random-access Image-dissector camera 
or a picture stored as a disk-file. The edge-marker is a non- 
linear parallel line-follower. That is, several lines are followed 
simultaneously, while the iBage Is being scanned in a raster-like 
fashion, thus combining the good features of raster-scan with those 
of line-following. The separation between scan-lines Is larger 
than the spacing of Image points sampled along a line. Three scan- 



lines are considered at any one time. The intensities are 
correlated with the three most common intensity transitions, namely 
the step, the roof and the peak. If any correlated value exceeds a 
threshold (adjusted according to the known slgnal-to-noise ratio), 
a check 1$ made for a local maximum. 

Once a likely edge-point (also called feature-point or 
lnhomogeneous point) is discovered, a subroutine checks to see if 
It could be the continuation of a line already being followed. The 
test involves a check on proximity and angle as well as such 
attributes as type, direction and size of intensity transition. If 
the edge-point cannot form the continuation of an existing line, 
yet is strong and not too close to any other line, it will be used 
as the starting point of a new line. This nev line will also be 
followed on subsequent scans. 

■ 

A line that cannot be continued in this manner is retained 
only if Its length exceeds some minimum. One pass over the scene 
consisting of successive horizontal lines picks up all edges within 
about fifty degrees of vertical. A second pass consisting of 
vertical lines does the same for edges within about fifty degrees 
of horizontal. For a typical scene about a million Intensity 
values are read from the lnage, and the whole process takes a few 

k 

minutes. 



Ve end up with a number of lists of tentative edge-points. 
Some of these lists will represent more than one edge ten L-Joint 
for example), while some edges will be represented by more than one 
list (a fragmented edge for example). These lists will often not 
Include points very close to vertices, except at L-joints. This Is 
because lines are prevented from encroaching too closely on each 
other by the nature of the correlation process and some of the 
tests described. Each list of tentative edge-points has associated 
with it a number of attributes such as type, direction and size of 
the intensity transition and how sure the program is that the edge- 
fragments are not merely due to noise. 

Up to this stage not very many heuristics have to be invoked 
and consequently the possibility of corrupting thfr data is snail* 
Numerous line-finders have been developed to this stage, few 
however produce as clean a set of tentative edge-points as T. 
Blnford's program. Very few line-finders have proceeded beyond 
this level to actually create the kind of useful cleaned-up line- 
drawing required by current higher-level scene-analysis programs. 
In part this is often due to inadequate edge-narking, but more 
often it is because unexpected difficulties are net when dealing 
with what at first sight seems a simple process: forcing the data 
into the form of a line-drawing with well-defined vertices. 
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4j_ GEHERATIHG LINES, 

Since some lists nay contain feature points of nore than one 
edge, the lists have to be segmented. This is done recursively At 
the point of maximum distance from the line connecting the end- 
points of the list of edge-points. Segmentation proceeds until the 
remaining edge-points fall within a tolerance band. Re- 
segmentation is used to cope with portions that are parallel to the 
line Joining the end-points. Once segmented, least-squares lines 
are fitted to the lists [Horn 1972] (The partial results of this 
fit continue to be part of the data associated with the line, to 
allow combining lines later without loss of accuracy). 

We next have to combine partial lines corresponding to the 
same edge. Overlapping lines are the first to be considered In 
this process, A number of tests are applied to avoid combining 
unrelated lines. These tests Include checks on proximity, relative 
orientation, perpendicular distance of the end-points from the 
potential combined line and so on. Lines more or less co-linear 
are combined in a similar manner provided the gap between them is 
relatively small. Any short lines remaining at this stage are 

discarded. 

■ 

The line-drawing is fairly recognizable at this stage, lacking 
only vertices. The lines have been distorted very little In this 
process, unless unrelated lines happened to be combined despite the 



stringent tests. The less conservative and nore difficult part is 
yet to come- The data at this stage is quite compact already and 
■ight profitably be handed to a program with some understanding of 
blocks-world scenes instead of the somewhat arbitrary heuristic 
program described next. 

5^ GENERATING VERTICES. 

Some of the vertices are clearly indicated by the close 
convergence of lines. A first estimate of the location of such a 
vertex is made by considering the center of gravity of the end- 
points of the lines (if there are only two lines, their 
intersection is used instead). A search is then made for all lines 
which end near this point and whose extension would pass very close 
to It* The vertex Is finally declared at the point of least- 
squares perpendicular distance from these lines [Horn 19721. The 
appropriate lines are then connected to the vertex. 

Next, the program tries to establish T-Joints. That Is, It 
searches for lines that have an end-point close to another line and 
checks if It would we reasonable to extend the line to form a new 
vertex. A K-Joint is similarly made where a vertex is close to a 
line. Finally, crossing lines give rise to X-joints, Numerous 
heuristics inform this process; for example, lines already 
connected at one end are treated preferentially to those still free 
at both ends. All vertices have now been found and an attempt Is 






made to extend unattached lines to nearby vertices. The extension 
must be short and pass very close to the vertex to be accepted. 
Following this, vertices which are close together are 
conglomerated. Any lines not properly attached at this stage are 
discarded. The progress of these steps In the processing can be 
slowed down and viewed on a display, before the final product is 
disgorged in a convenient LISP-readable format. 

The heuristics, which developed empirically, depend on certain 
tolerances which are Initially calculated in terns of the line-scan 
Interval, the known resolution of the imaging device, and the 
signal-to-noise ratio- These factors could be "tuned" to improve 
performance and accuracy, but this is probably not worth the effort 
(the time would be better spent on designing new ways of directing 
the vertex creation phase). A large portion of this assembly 
language progran is concerned with debugging, displaying and 
performing the required list-processing. The data-base Is 
maintained in two forms: As assertions about lines (and what 
vertices they connect to) and as assertions about vertices (and 
what lines impinge on them). These two forms are kept consistent 
by demons Invoked vnen assertions are added, removed or changed In 

either data-base. 

■ 

The line and vertex generating phases of the program 
frequently need to settle questions of proximity for both points 
and lines. A set of four superimposed rectangular grids covering 
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the image Is used as a set of buckets for this purpose. This 
method Is sometimes referred to as multi-entry coding [Horn 1972]. 
Each point Is entered into four buckets, and each line will appear 
In many- This makes for high speed, despite the need for a number 
of iterative applications of various heuristics to the whole data- 
structure. 

k_ FERFPRHANCfit 

An Idea about how to produce line-drawings from images Is of 
little use until it has been demonstrated by a working program. 
The program discussed here has processed over a hundred scenes and 
produces excellent line-drawings of simple blocks-world scenes. In 
more complicated cases, a number of shortcomings CU be observed. 
The simplest and easiest to deal with is the absence or 
Incompleteness of some of the lines, usually due to a lack of 
contrast between adjacent faces of an object. At times extra lines 
are introduced by shadows, smudges and noise. There is a trade-off 
between these two effects, and since present scene-analysis 
programs can handle missing lines better than extra lines, the 
threshold Is set to favour the former. 

Occasionally a section of a line-drawing will be garbled, 
usually due to the combination of two unrelated vertices. This 
causes some distortion of the lines and Bay make the line-drawing 
locally uninterpretable. This last effect In particular Is a 



function of the amount of detail and can be avoided by using a 
finer resolution at the cost of an increase in scan and computation 
time. This of course is only possible if the imaging device has 
sufficient resolution. 

2s_ COMMENTS ON THIS APPROACH. 

Soie of the ways in which images of sets of polyhedra differ 
from our Intuitive model of equal intensity polygonal areas have 
Important implications in other areas. Mutual Illumination for 
example will prove to be a problem when one is developing a program 
that exhibits color constancy. Further, it should be noted that 
some of the edges missed by this program will also be missed by 
nne-verifUrs; when follpwing lines one can afford to be quite 
sensitive. Fortunately the better line-proposers are very 
conservative and hardly ever propose a line were there shouldn't be 
one. One could perhaps accept their proposals without attempting 
verification. 

The only features that can be reliably determined from a 
corrupted image are those with significant spatial extension. 
Without such extension we cannot apply the Integrative processes 
necessary to collect evidence for the existence of the feature. 
Vertices for this reason are perhaps not primitive elements of an 
image, but exist only as the intersection of lines. Letting the 
line-drawing program establish vertices Introduces inaccuracies, 



because It may join unrelated lines. 



B. HOW TO DO BETTER. 



This program has no Idea what a reasonable line-drawing should 
look like when It represents an image of polyhedra. Instead It Is 
very general and will find arbitrary line-drawings. Observing the 
particular way In which things sometimes go wrong, one quickly 
comes to the conclusion that higher-level understanding of the 
scene being analysed could greatly inprove the line and vertex 
creating phase of this progran. As things stand now this 
understanding comes only after the line-finder has done Its work. 
It would be hard to embed this kind of knowledge Into such a large 
assembly language program. At the same time trying to lnplenent 
the "low-level" routines In a higher level language would cause 
great Inefficiencies. 

w 

&_ H15TQRICAL NOTE. 

W * 

The line-finder described here consists of an edge-marker due 
to T. Blnford and a line-drawer due to B. K. P. Horn. It was used as 
part of the system of vision and manipulation programs developed 
for the copy-demo by P.H. Winston, E. Freuder and B. K. P. Horn In 
the fall of 1970 [Winston 1971 6 1972]. It Is also the 
"hierarchical" program referred to by Shirai [Shiral 19733- 
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